Journals
  Publication Years
  Keywords
Search within results Open Search
Please wait a minute...
For Selected: Toggle Thumbnails
Cross-layer data sharing based multi-task model
Ying CHEN, Jiong YU, Jiaying CHEN, Xusheng DU
Journal of Computer Applications    2022, 42 (5): 1447-1454.   DOI: 10.11772/j.issn.1001-9081.2021030516
Abstract286)   HTML28)    PDF (1841KB)(105)       Save

To address the issues of negative transfer and difficulty of information sharing between loosely correlated tasks in multi-task learning model, a cross-layer data sharing based multi-task model was proposed. The proposed model pays attention to fine-grained knowledge sharing, and is able to retain the memory ability of shallow layer shared experts and generalization ability of deep layer specific task experts. Firstly, multi-layer shared experts were unified to obtain public knowledge among complicatedly correlated tasks. Then, the shared information was transferred to specific task experts at different layers for sharing partial public knowledge between the upper and lower layers. Finally, the data sample based gated network was used to select the needed information for different tasks autonomously, thereby alleviating the harmful effects of sample dependence to the model. Compared with the Multi-gate Mixture-Of-Experts (MMOE) model, the proposed model improved the F1-score of two tasks by 7.87 percentage points and 1.19 percentage points respectively on UCI census-income dataset. The proposed model also decreased the Mean Square Error (MSE) value of regression task to 0.004 7 and increased the Area Under Curve (AUC) value of classification task to 0.642 on MovieLens dataset. Experimental results demonstrate that the proposed model is suitable to improve the influence of negative transfer and can learn public information among complicated related tasks more efficiently.

Table and Figures | Reference | Related Articles | Metrics